Decision Maker using Coupled Incompressible-Fluid Cylinders

The multi-armed bandit problem (MBP) is the problem of finding, as accurately and quickly as possible, the most profitable option from a set of options that gives stochastic rewards by referring to past experiences. Inspired by fluctuated movements of a rigid body in a tug-of-war game, we formulated a unique search algorithm that we call the ‘tug-of-war (TOW) dynamics’ for solving the MBP efficiently [1-5]. The cognitive medium access, which refers to multiuser channel allocations in cognitive radio, can be interpreted as the competitive multi-armed bandit problem (CMBP); the problem is to determine the optimal strategy for allocating channels to users which yields maximum total rewards gained by all users [6]. Here we show that it is possible to construct a physical device for solving the CMBP, which we call the ‘TOW Bombe’, by exploiting the TOW dynamics existed in coupled incompressible-fluid cylinders. This analog computing device achieves the ‘socially-maximum’ resource allocation that maximizes the total rewards in cognitive medium access without paying a huge computational cost that grows exponentially as a function of the problem size.